Vehicular imaging system with misalignment correction of camera

ABSTRACT

A vehicular imaging system includes a camera disposed behind a windshield of a vehicle and viewing through a portion of the windshield. Image data captured by the camera is provided to a control. The control receives, via a communication bus of the vehicle, at least one selected from the group consisting of (i) vehicle pitch information relating to pitch of the vehicle, (ii) vehicle yaw information relating to yaw of the vehicle and (iii) vehicle steering information relating to steering of the vehicle. The system automatically corrects for misalignment of the camera. Image data captured by the camera is processed at the control for a lane departure warning system of the vehicle and for at least one selected from the group consisting of (i) an automatic headlamp control system of the vehicle, (ii) a collision avoidance system of the vehicle and (iii) an adaptive front lighting system of the vehicle.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/947,270, filed Jul. 27, 2020, now U.S. Pat. No. 11,328,447, which isa continuation of U.S. patent application Ser. No. 15/978,435, filed May14, 2018, now U.S. Pat. No. 10,726,578, which is a divisional of U.S.patent application Ser. No. 14/694,226, filed Apr. 23, 2015, now U.S.Pat. No. 9,972,100, which is a continuation of U.S. patent applicationSer. No. 13/776,094, filed Feb. 25, 2013, now U.S. Pat. No. 9,018,577,which is a continuation of U.S. patent application Ser. No. 13/204,791,filed Aug. 8, 2011, abandoned, which is a continuation of U.S. patentapplication Ser. No. 12/190,698, filed Aug. 13, 2008, now U.S. Pat. No.8,017,898, which claims the benefit of U.S. provisional application Ser.No. 60/956,633, filed Aug. 17, 2007, which are incorporated herein byreference for all purposes.

FIELD OF THE INVENTION

The present invention relates to automatic headlamp control systems forvehicles and, more particularly, to automatic headlamp control systemsthat automatically adjust the high and low beam states of a vehicleheadlamp.

BACKGROUND OF THE INVENTION

Automotive forward lighting systems are evolving in several areasincluding the use of image-based sensors, typically referred to asAutomatic High Beam (AHB) control systems, to maximize the use of highbeam road illumination when appropriate, the use of steerable beamsystems, typically referred to as Adaptive Front Lighting (AFL) systems,to provide a greater range of beam pattern options particularly fordriving on curved roads or during turn maneuvers wherein the beampattern may be biased or supplemented in the direction of the curve orturn, and the combination of such AHB and AFL systems.

Automatic high beam control system are known that utilize an opticalsystem, an image sensor, and signal processing including spectral,spatial and temporal techniques to determine ambient lightingconditions, the road environment, and the presence of other road usersin order to automatically control the selection of the appropriateforward lighting state such that user forward vision is optimized whileminimizing the impact of headlamp caused glare on other road users inall lighting conditions. Examples of such systems are described in U.S.Pat. Nos. 5,796,094; 6,097,023; 6,320,176; 6,559,435; 6,831,261;6,396,397; 6,822,563 and 7,004,606, which are hereby incorporated hereinby reference in their entireties.

While AHB systems that utilize the features and concepts describedwithin the above identified U.S. patents have achieved performancelevels that have resulted in considerable commercial success, it isdesired to provide additional features and techniques, which mayincrease the utility, improve the performance, facilitate themanufacture, and simplify the installation of such systems.

SUMMARY OF THE INVENTION

The present invention provides an automatic headlamp control system thatis operable to automatically control or adjust the high beam state of avehicle's headlamps. The headlamp control system is operable todetermine a focus of expansion pixel or pixels in the captured image andadjust the image processing in response to the location or pixel/pixelsof the focus of expansion and the tracking of movement of detected lightsources and other objects as the vehicle travels along the road. Theheadlamp control system of the present invention may provide reducedprocessing of image data to provide a low cost system.

According to an aspect of the present invention, a vehicular imagingsystem comprises a photosensor array comprising a plurality ofphotosensor elements and a control responsive to an output of thephotosensor array. The photosensor array has a field of view forward ofthe vehicle that is generally in line with the vehicle's primarydirection of forward travel. The photosensor array captures images of anarea encompassed by the forward field of view. The control processes animage data set indicative of captured images. The control processes areduced image data set of the image data set to determine whether anobject of interest is within a target zone of the captured images. Thereduced image data set is representative of a portion of the capturedimages as captured by a particular grouping of the photosensor elements.Responsive to a determination of a change in a focus of expansion of thecaptured images, the control adjusts the reduced image data set so as tobe representative of a portion of the captured images as captured by adifferent particular grouping of the photosensor elements.

The control may be operable to adjust a state of a headlamp beam inresponse to the image processing. The focus of expansion comprises atleast one photosensor element that initially detects a new light sourcein the field of view. The control may track the new light source as itexpands in the captured images (such as while the relative distancebetween the controlled vehicle and the new light source decreases) toconfirm that the new light source is indicative of an object ofinterest. The control may determine that the new light source isrepresentative of a light source of a leading or approaching vehicle andthe controlled vehicle and approaching vehicle are traveling along asubstantially flat and substantially straight road, and the control maycompare a location of the new light source (such as when it is at ornear the targeted zone of interest) to an expected location of the lightsource to determine if there is an offset. The control may process manysamples of new light sources to arrive at an optimal or enhanced offset.The control adjusts the reduced data set in response to determination ofsuch an offset.

According to another aspect of the present invention, a vehicularimaging system includes a photosensor array having a plurality ofphotosensor elements and a control responsive to an output of thephotosensor array. The photosensor array has a field of view forward ofthe vehicle that is generally in line with the vehicle's primarydirection of forward travel. The photosensor array captures images of anarea encompassed by the forward field of view. The control processesimage data indicative of captured images, and is operable to selectivelyprocess the output of the photosensor array as an output from thephotosensor array at two or more different resolutions. The controlutilizes a single classifying parameter for identifying a particularobject of interest in the forward field of view for all of the at leasttwo resolutions.

Optionally, for example, the at least two different resolutions maycomprise (a) an output of a higher resolution photosensor array, (b) anoutput of a medium resolution photosensor array, and (c) an output of alower resolution photosensor array. The control may process the outputof the photosensor array at different resolutions in response to one of(a) a location of a detected light source and (b) a distance between thesubject vehicle and a detected light source.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side elevation of a portion of a vehicle embodying thepresent invention;

FIG. 2 is a partial side elevation view and block diagram of a vehicleheadlight dimming control system according to the present invention;

FIG. 3 is a schematic of an imaging array suitable for use with thecontrol system of the present invention;

FIG. 4 is a schematic of a determination of an offset of a focus ofexpansion for the control system of the present invention; and

FIG. 5 is a schematic of a headlamp control system utilizing a cellphone camera for capturing images of a forward field of view.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an automatic vehicle headlamp controlsystem or vehicle headlamp dimming control system 12, which includes animage sensor 14 which senses light from a scene forward of vehicle 10,an imaging processor or control circuit 13 which receives data fromimage sensor 14 and processes the image data, and a vehicle lightingcontrol logic module 16 which exchanges data with control circuit 13 andcontrols the headlamps 18 (such as by changing or retaining the state ofthe headlamps, such as between a higher beam state and a lower beamstate) of vehicle 10 for the purpose of modifying the beam illuminationstate of the headlamps of the vehicle (FIGS. 1 and 2). The headlamps areoperable to selectively emit a light output via a high beam lightingelement and a lower beam or low beam lighting element. Headlamp dimmingcontrol 12 is operable to determine whether light sources in the imagecaptured by the image sensor are or may be indicative of headlamps ofoncoming vehicles or taillights of leading vehicles and is operable toadjust the headlamps of the controlled vehicle between a high beam stateand a lower beam state or low beam state in response to such adetermination. Headlamp dimming control 12 may utilize the principlesdisclosed in U.S. Pat. Nos. 5,796,094; 6,097,023; 6,320,176; 6,559,435;6,831,261; 6,396,397; 6,822,563 and/or 7,004,606, which are herebyincorporated herein by reference in their entireties. Headlamp control12 is operable to distinguish the light sources captured in the imagebetween light sources representing headlamps and/or taillights of othervehicles, as discussed below.

The imaging sensor for the headlamp control of the present invention maycomprise any suitable sensor, and may utilize various imaging sensors orimaging array sensors or cameras or the like, such as a CMOS imagingarray sensor, a CCD sensor or other sensors or the like, such as thetypes described in U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962;5,715,093; 5,877,897; 6,498,620; 5,796,094; 6,097,023; 6,320,176;6,559,435; 6,831,261; 6,806,452; 6,396,397; 6,822,563; 6,946,978;7,038,577 and/or 7,004,606; and/or U.S. patent application Ser. No.11/315,675, filed Dec. 22, 2005 and published Aug. 17, 2006 as U.S.Patent Publication No. US-2006-0184297; and/or U.S. provisionalapplications, Ser. No. 60/845,381, filed Sep. 18, 2006; and Ser. No.60/837,408, filed Aug. 11, 2006; and/or PCT Application No.PCT/US2007/075702, filed Aug. 10, 2007, and published Feb. 28, 2008 asPCT Publication No. WO 2008/024639, and/or PCT Application No.PCT/US2003/036177, filed Nov. 14, 2003, and published Jun. 3, 2004 asPCT Publication No. WO 2004/047421, which are all hereby incorporatedherein by reference in their entireties. The control 12 may include alens element or optic 20 between the image sensor and the forward sceneto substantially focus the scene at an image plane of the image sensor.Optionally, the optic may comprise an asymmetric optic, which focuses agenerally central portion of the scene onto the image sensor, whileproviding classical distortion on the periphery of the scene or field ofview.

Such imaging sensors or cameras are pixelated imaging array sensorshaving a photosensing array 15 of a plurality of photon accumulating orphotosensing light sensors or pixels 15 a (FIG. 3), which are arrangedin a two-dimensional array of rows and columns on a semiconductorsubstrate. The camera established on the substrate or circuit boardincludes circuitry which is operable to individually access eachphotosensor pixel or element of the array of photosensor pixels and toprovide an output or image data set associated with the individualsignals to the control circuit 13, such as via an analog to digitalconverter (not shown). As camera 14 receives light from objects and/orlight sources in the target scene, the control circuit 13 may then beoperable to process the signal from at least some of the pixels toanalyze the image data of the captured image, as discussed below.

As shown in FIG. 3, the control may process one or more sub-arrays 15 bof the photosensor array 15, where a particular sub-array may berepresentative of a zone or region of interest in the forward field ofview of the camera. The control may process the sub-array of pixelswhile ignoring other pixels or processing other pixels at a reducedlevel (such as by utilizing aspects of the systems described in U.S.Pat. No. 7,038,577, which is hereby incorporated herein by reference inits entirety), and/or the control may process the sub-array of pixels ina particular manner (such as to determine if a light source is a vehiclelamp in the regions forward of the vehicle and near the host vehicle'spath of travel, such as a headlamp of an oncoming vehicle in a laneadjacent to (such as to the left of) the host vehicle or other vehiclelamp forward and/or to the left or right of the host vehicle) whileprocessing other sub-arrays or pixels in a different manner.

In order to take advantage of the environmental protection offered bythe vehicle cabin, the frequently cleaned optically clear path offeredby the vehicle windshield (which is cleaned or wiped by the windshieldwipers when the wipers are activated), and the relatively high vantagepoint offered at the upper region or top of the windshield, the headlampcontrol system or at least the imaging device or camera is preferablymounted centrally at or near the upper inside surface of the frontwindshield of a vehicle and with a forward field of view through theregion cleaned or wiped by the windshield wipers. The imaging device maybe mounted at an interior rearview mirror assembly (such as at amounting bracket or base of the mirror assembly) or at an accessorymodule or windshield electronics module disposed at or near the interiorrearview mirror assembly and at or near the interior surface of thevehicle windshield.

Automatic image-based high beam control systems, in which an image ofthe scene forward of the vehicle is focused by an optical system, mayhave a horizontal field of view equal to, but not limited to,approximately +/−24 degrees about the imaging system centerline. Thishorizontal field of view may be larger than (and may be substantiallylarger than) the horizontal extent of the high beam pattern, butoptionally the high beam pattern itself may be moved left and right upto approximately 15 degrees in either direction by an adaptive frontlighting (AFL) system. The image may be focused or imaged onto arectangular array image capture device, such as, but not limited to,onto a 640×480 CMOS color imager, which captures image data and providessequential frames of data indicative of the light energy reflected oremitted by objects in the region subtended by each element of the array.The image capture rate may be at a rate in the range of about 5 to 120times per second or more, with processing being performed on the data todetermine the presence, location and characteristics of objects and/orlight sources within the monitored scene and to determinecharacteristics of the monitored scene, such as general illuminationlevel, and to utilize several defined regions or zones of the monitoredscene for various purposes. For example, the region of the scene thatgenerally corresponds to the region of influence of the vehicle highbeam pattern may be used to determine the appropriate high beam state ofthe headlamps depending on whether or not other road users are detectedwithin that region. Optionally, the regions to the left and right of thefirst region may be used to anticipate the upcoming entry of other roadusers into the first region in order to facilitate a rapid andappropriate response upon entry or just prior to entry of the firstregion. The upper central region of the monitored scene may be used todetermine ambient lighting conditions such that a first threshold may beestablished below which low beam headlights are activated, and a secondthreshold may be established above which high beam activation may beinhibited, while the lower horizontal portion of the ambient lightingcondition detection region may be used to detect urban lightingconditions or the like. Other processing of the captured image data maybe implemented depending on the particular application of the imagesensor and processor, while remaining within the spirit and scope of thepresent invention.

The control system of the present invention thus captures images orgenerates image data indicative of the scene occurring forwardly of thevehicle and processes the image data to determine whether or not aheadlamp or taillight of another vehicle is present, whereby theheadlamps of the controlled vehicle may be adjusted between their highand low beams accordingly. The image processor processes one or morezones of interest or regions of interest to determine whether thedetected light source is a headlamp or taillight of another vehicletraveling on the same road as the controlled vehicle (since a lightdetected that is remote from the particular or appropriate region ofinterest is not likely another vehicle light or is not likely relevantto the decision as to whether or not the controlled vehicle's headlampsshould be adjusted). The control system thus may utilize aspects of theimage systems and/or headlamp control systems described above to processdifferent zones or regions of interest, and may ignore other regions ofthe captured images or process other regions at a reduced level (such asby utilizing aspects of the systems described in U.S. Pat. Nos.5,550,677; 5,877,897 and/or 7,038,577, which are hereby incorporatedherein by reference in their entireties).

In order to ensure that the region of interest or regions of interestbeing processed are representative of the appropriate region relative tothe controlled vehicle and direction of travel thereof, the controlsystem of the present invention is operable to provide an automaticalignment or correction factor of the image data captured by the imagesensor. Thus, the various regions of interest within the scene monitoredby the sensor are optimally maintained regardless of vehicle and highbeam control system module geometric manufacturing and assemblytolerances, and other sources of misalignment, such as vehicle pitch andyaw variations due to a wide range of possible vehicle loadingconditions.

Typical vehicle body structures, windshields and assembly systems ofvehicles may contribute to geometric tolerances associated with thesurface to which the headlamp control system module is attached. It isnot unusual to encounter a total stack up of tolerances which result ina potential vertical and horizontal misalignment of approximately +/−4degrees from the theoretically ideal condition. This is a significantvalue and may result in errors in processing the appropriate region ofinterest and/or determining lane widths and object sizes and distancesand the like.

It is known to provide a mechanical adjustment means to allow for thecorrection of such a misalignment at the installation of a headlampcontrol system to the vehicle. Such mechanical adjustments are, however,often undesirable since it is often expensive to apply manual labor tothe alignment of components on each vehicle equipped with a headlampcontrol system at the vehicle assembly plant or facility. Suchadjustments are additionally undesirable since the alignment procedureis then subject to operator error.

Also, such adjustment will only correct for misalignment of the imagingdevice and system at the time of manufacturing of the vehicle, and willnot correct or account for or adapt the system for misalignment that mayoccur during use, such as due to a physical or mechanical misalignmentof the imaging device or due to different load balancing of the vehicleor replacement of the camera or mirror assembly or assembly module orwindshield and/or the like. For example, in normal use, a typicalvehicle experiences many different loading conditions which cause it toadopt a wide range of pitch and roll attitudes, causing an automaticheadlamp control system of the vehicle to view the forward scene fromperspectives different from the ideal, or initially considered designconditions, and thereby potentially resulting in different headlightactuation decisions than contemplated by the original systemspecification.

Thus, it is beneficial for the headlamp control system to include afeature which automatically compensates for an initial misalignmentcondition and additionally is capable of correcting for temporaryvehicle conditions and re-installation misalignments which may occurduring the use of the vehicle. In order to achieve optimum performanceof the headlamp control system, it is desirable to determine which ofthe array elements of the image capture device fall into each of thedefined regions of interest. Since the regions are defined relative tothe forward scene, it is desirable to determine a particular point orarea within the forward scene and to relate that point or area to aparticular array element or photosensor or group of photosensors of theimage capture device.

The particular point in the forward scene may be defined as a particulardistant point or area which lies on the forward extended vehiclecenterline on the horizontal plane which passes generally through thecenter of the optical system associated with the image capture device.When driving on a substantially flat and substantially straight road,the distant point may be the point within the forward scene at which theheadlights of an oncoming vehicle or the tail lamps of a slower leadingvehicle are first detected. As the distance between the controlledvehicle and target vehicle decreases, the image of the target vehicleexpands within the imaged scene, towards the left if traveling in aleftward lane, centrally if in the same lane, and towards the right iftraveling in a rightward lane. Thus, the described distant point may becalled the focus of expansion or FOE.

In order to determine the imaging array element or pixel which subtendsthe FOE in the as assembled and as loaded vehicle, it is necessary toidentify the array element or pixel or pixels which first detects a newlight source (which has the potential to be a vehicular light source orheadlamp or taillight) within that region of the monitored scene whichcould potentially contain the FOE, and to continue to track the detectedlight source as it expands in the image as the distance between thedetected source and the controlled vehicle decreases until it isconfirmed that the source is a headlamp or taillight of another vehicle(such as by utilizing aspects of the systems described in U.S.provisional applications, Ser. No. 60/845,381, filed Sep. 18, 2006; andSer. No. 60/837,408, filed Aug. 11, 2006, and/or PCT Application No.PCT/US2007/075702, filed Aug. 10, 2007, and published Feb. 28, 2008 asPCT Publication No. WO 2008/024639, which are hereby incorporated hereinby reference in their entireties). The control system may monitor thecontrolled vehicle trajectory until it reaches the point in the roadwhere the new light source would have been initially detected in orderto confirm that the road traveled for the duration of the monitoringperiod was substantially flat and substantially straight. If it isdetermined that the point or light source was a light source of aleading or approaching vehicle and the controlled vehicle andapproaching vehicle are traveling along a substantially flat andsubstantially straight road, the location of the initial distant pointor FOE may be compared to an expected location (the location of thepixel corresponding to the preset or expected FOE) to determine if thereis an offset or error in the imaging device's or system's calibration.The control system optionally, and preferably, collects or processes oranalyzes many new light sources and picks the best samples and averagesthem to arrive at the best or optimal or enhanced FOE.

If an offset between the actual or detected FOE and the expected orpreset FOE is detected, the image processor determines the degree ofoffset and adjusts or shifts the regions of interest parameters orcoordinates or targeted pixels to accommodate for the offset, such thatthe processor processes the image data captured by the pixelsrepresentative of the appropriate zones or regions of interest forwardlyof the controlled vehicle for detecting headlamps of approachingvehicles and taillights of leading vehicles. For example, if thedetected FOE is ten pixels to the left and five pixels down from theexpected FOE, the processor may readily adjust the parameters orcoordinates of the regions of interest by that amount (or by a scaledvalue based on the detected offset). Thus, the headlamp control systemmay adjust the processing to adapt to shifts or changes in the FOE ofthe imaging device and thus may do so electronically and withoutphysical or mechanical adjustment of the imaging device relative to thevehicle.

The headlamp control system of the present invention thus provides a lowcost processing adjustment to maintain processing of the appropriateregions of interest when detecting light sources or objects forwardly ofthe vehicle and determining whether or not the detected light sources orobjects are leading vehicles or approaching vehicles along the road onwhich the controlled vehicle is traveling. The control system thuscalibrates or adapts the image data or image processing to accommodatefor manufacturing tolerances and/or physical misalignment that may occurduring the camera and/or mirror or accessory module manufacture orduring the vehicle manufacture, and to accommodate for misalignment orshifts in the principal viewing axis of the camera or imaging device dueto different load balancing of the vehicle or distortion in shape of theheadlamp control system assembly due to heating and/or other situationswhere the vehicle encounters or experiences a change in pitch or tilt oryaw of the vehicle.

Optionally, the control system may be adjusted in response to adetection of lane markers, such as along a straight and/or flat road (oroptionally along a curved road and optionally in conjunction with asteering angle of the vehicle). For example, and with reference to FIG.4, the system may detect lane markers 22 along the lane in which thecontrolled vehicle is traveling and, if the lane markers aresubstantially straight, may determine an intersection 22 a of the lanemarkers. The control system may detect headlamps in front of the vehicleand may monitor the appearance point 24 of the detected headlamps. Thesystem may monitor the appearance point and the intersection point as ittravels toward and past the physical locations corresponding to themonitored points in the captured images, and if it is determined thatthe road or lane was substantially straight, the offset between theactual FOE and the expected FOE may be determined. The control systemcollects determined lane-marker-based FOE samples and rejects the errorsamples and averages them to arrive at a best or optimal or enhancedlane-marker-based FOE.

Optionally, the control system may utilize a weighted sum calculation ofdata representative of the intersection point and the (appearance pointplus offset) to determine the actual FOE, depending on the particularapplication. For example, the adaptive FOE may be based on a detectionof the appearance (initial detection) of lights and disappearance oflights (when the lights are out of the range of the sensor and are nolonger detected) in front of the vehicle and a detection of the lanemarkers along the road in front of the vehicle, and may be calculated,for example, via the following equations:

AFOE_(ROW)=(a*[LaneMark Row+Offset]+b*[Headlight AppearRow+Offsetv]+c*[Taillight Disappear Row+Offsetw])/(a+b+c); and   (1)

AFOE_(COLUMN)=(d*[LaneMark Column+Offsetx]+e*[Headlight AppearColumn+Offsety]+f*[Taillight Disappear Column+Offsetz])/(d+e+f);   (2)

where a, b, c, d, e and f are parameter weights that depend on theparticular application. Other equations may be utilized to substantiallyestimate or calculate the present FOE of the imaging device, such asbased on the detection of lane markers and/or light sources and/or thelike in the forward field of view. Since this method uses either or bothlane markers or vehicle appearance/disappearance, the system can workfor environments without lane markers or for environments withoutinitial other-vehicle presence.

Optionally, the control system may be adjusted in response to vehiclepitch information from a bus or accelerometer, and/or vehicle rollinformation from an accelerometer or bus information of the vehicle,and/or vehicle yaw information from an accelerometer or bus informationof the vehicle. Optionally, the system may only monitor for new lightsources when the vehicle is traveling in a substantially straight line(such as when the steering wheel angle is between, for example, about0+/−10 degrees for a vehicle with steering ratio of about 17, or at orbetween any other suitable or selected threshold angle or anglesdepending on the particular application of the control system). Thus,adjustment and/or alignment of the image sensor may occur by trackingmovement of light sources through the images when the vehicle istraveling substantially straight, so that the control may compare thetracked light sources to expected locations and paths through thecaptured images as the vehicle moves along the substantially straightpath and may adjust the processing parameters of the image processor andimaging sensor accordingly.

Optionally, the control system may determine the actual FOE and offsetalong curved road sections in response to the lane marker detectionand/or a steering angle input, whereby the system may monitor thedetected appearing light source and monitor its initial or appearancelocation as the controlled vehicle approaches the initial location. Bytaking into account the steering angle of the vehicle as the vehicletravels toward the initial or appearance location of the light source,the control system may monitor or track the initial location todetermine if the controlled vehicle approaches or arrives at or nearthat location. The control system may also determine if the detectedlight source was a headlamp of an approaching vehicle or taillight of aleading vehicle and, if so, may determine the offset and adjust or adaptthe image processing accordingly.

The automatic adjustment or correction or adaptation of the imageprocessor in response to a detected offset between a detected FOE and anexpected FOE allows the control system of the present invention toutilize various cameras or imaging devices, such as aftermarket devicesor cell phone cameras or the like. For example, an aftermarket cameramay be installed in the vehicle with a generally forward field of viewin the direction of travel of the vehicle, and the system may, as thevehicle is then driven, determine an offset or error in the expected FOEand readily compensate for such offset, without requiring any furthermanual input or physical adjustments.

Thus, it is envisioned that any imaging device (such as, for example, acell phone camera) may be utilized for the imaging system or headlampcontrol system of the present invention. For example, and with referenceto FIG. 5, a cell phone 26 may be docked at (such as at a phoneconnector mount or port 28 or the like at the vehicle instrument panelor dashboard 29 or mirror assembly or accessory module or console or thelike) or in communication with an image processor that processes theimages captured by the cell phone camera 26 a in a similar manner asdescribed above, and determines the current FOE for the cell phonecamera at its present orientation relative to the vehicle and determinesthe appropriate zones of interest or regions of interest for processingthe image data to determine if detected light sources in the forwardfield of view are representative of a headlamp of an approaching vehicleor taillight of a leading vehicle. The cell phone may transmit acompressed video stream (such as, for example, at about 264 Hz) to acell phone network, and/or may communicate video signals to an on-boardor vehicle-based processor.

It is further envisioned that the adaptive FOE process of the presentinvention allows for the use of various aftermarket cameras and/or cellphone cameras for various imaging systems or applications, such asadaptive front lighting systems or lane departure warning systems orobject detection systems or collision avoidance systems or the like,since the camera (such as a cell phone and camera) may be located at orin or mounted at the vehicle and the processing of the image data may beadapted to automatically accommodate for and correct for anymisalignment or mis-mounting or mis-positioning of the camera.Optionally, aspects of the adaptive FOE system described above may beutilized for cameras (such as OEM cameras or aftermarket cameras or cellphone cameras or the like) having a rearward field of view so that theprocessing of the captured images is corrected or adapted and the imagesare processed accordingly, such as for a rear vision system or backupaid or the like, and/or may be utilized for cameras having a field ofview directed inside the vehicle, such as for interior cabin monitoringsystems or the like (such as utilizing aspects of the systems describedin U.S. Pat. Nos. 5,760,962; 5,877,897 and/or 6,690,268, which arehereby incorporated herein by reference in their entireties). The imagedata from the cell phone camera (or other camera) may be communicatedwirelessly (such as via a short-range radio frequency communication,such as via a BLUETOOTH® communication protocol or the like) or via awired connection (such as via a docking port or USB port or the like atthe vehicle) to a vehicle-based or onboard processor (such as processor13 described above), or compressed video data or image output of thecamera may be streamed to a cell phone network or the like.

Optionally, the control system may adjust the zones of interest orregions of interest in the captured images in response to an inputrepresentative of the vehicle trajectory, such as in response to asteering angle of the vehicle or steering wheel angle of the vehicle,such as by utilizing aspects of the systems described in U.S. patentapplication Ser. No. 11/315,675, filed Dec. 22, 2005, and published Aug.17, 2006 as U.S. Publication No. US-2006-0184297; and/or U.S.provisional applications, Ser. No. 60/845,381, filed Sep. 18, 2006; andSer. No. 60/837,408, filed Aug. 11, 2006, and/or PCT Application No.PCT/US2007/075702, filed Aug. 10, 2007, and published Feb. 28, 2008 asPCT Publication No. WO 2008/024639, which are all hereby incorporatedherein by reference in their entireties. For example, when thecontrolled vehicle is traveling along a curved road, the zones ofinterest or regions of interest may be adjusted or offset (i.e., theimage processor may process different groups of pixels corresponding tothe different zones or regions of interest) so that the image processorprocesses the adjusted or offset zones or regions of interest to detectvehicles along a curved road. Such an adjustment of the zones ofinterest parameters or pixel locations may occur when the control systemdetermines that the controlled vehicle is traveling along a curved road,such as in response to a steering angle input or lane detection input orthe like. Optionally, the regions of interest may be reduced or shrunk(or optionally extended) at the corresponding side regions depending onthe direction of the curve along which the vehicle is traveling.

Optionally, the control system of the present invention may adjust oradapt other processing parameters based on previous image processing.For example, the control system may process a large window or region ofinterest and may adapt the region of interest to a smaller region orwindow if a light source is detected. For example, if a detected lightsource is identified as headlamps of an approaching vehicle, the regionof interest may be adapted from a large region or zone or window to asmaller or predicted region that is representative of where theheadlamps of an approaching vehicle should be located relative to thecontrolled vehicle, such as down and to the left in the captured image(for a detected taillight, the adaptive region or window may begenerally downward at or near a center line of the image as thecontrolled vehicle approaches a leading vehicle). If the adapted orsmaller or inner or predicted region or window no longer detects a lightsource or object, the control system may resume processing of the largerregion of interest or window to determine if other light sources orobjects are present. Optionally, the exposure may be adjusted or adaptedfrom one frame to the next, such as if an approaching headlamp isdetected, for example, the exposure may be less for the subsequentframes as the headlamp moves closer to the controlled vehicle. Fordetected taillights, the change in exposure may be reduced or inhibited,since the taillights typically move generally with the controlledvehicle and do not approach the controlled vehicle as rapidly as toheadlamps of approaching vehicles.

The driving side of the road varies by countries of the world (forexample, in the United States, the vehicles are driven on the right sideof the road). It is desirable to support left and right driving side ofroad to reduce engineering and manufacturing cost. Thus, it isbeneficial for the head lamp control system to include a feature thatautomatically detects the driving side of the road. The control systemthus may be operable to process the image data set to detect the newlight source and identify the head light of the oncoming vehicle. Thecontrol system tracks the head light of the detected oncoming vehicleand stores the trajectory of headlight. The driving side of the vehicle(the side of the road along which the vehicle is traveling) isidentified by analyzing a pre-determined amount of the trajectories ofthe oncoming vehicles. If the driving side is on the right side of theroad, then the oncoming vehicles will pass the host vehicle on the left,and vice versa. Optionally, the control system may detect the drivingside only by analyzing the location of appearance of new headlights inimage data, since the appearance of new head light sources is biasedtoward one side of the captured image or image data according thedriving side of the road when the vehicle is traveling along asubstantially flat and straight road. Optionally, the control system maybe responsive to a global positioning system input and may determine thedriving side of the road on which the vehicle is traveling based on thegeographical location of the vehicle and the driving rules andregulations of that geographical location or region.

The automatic high beam system or automatic headlamp control system maybe optimized to adapt the vehicle for enhanced performance for theparticular road (and side of the road) along which the vehicle is beingdriven. For example, the control system may modify many calibrations orparameters such as, but not limited to, different zones or sub-arrays ofimage data, weighting factors of different zones of image data, offsetof the FOE, automotive light sources acceptance parameters, lane markerdetection and tracking of objects and light sources and/or the like, inorder to adapt the system for enhanced performance depending on whichside of the road the host vehicle is driven on.

Optionally, the control system of the present invention may be operableto provide a low-cost processing of the image data via processingcaptured frames of image data at different resolution levels, such as atleast two different resolutions or resolution levels. For example, thecontrol system may process images at a higher resolution level (wherethe imaging device may be processed, for example, as a pixelated arrayof 640×480 pixels), at a medium or intermediate resolution (where theimaging device may be processed, for example, as a pixelated array of320×240 pixels), and at a lower resolution (where the imaging device maybe processed, for example, as a pixelated array of 160×120 pixels). Sucha processing technique allows the processor to use the same classifier(such as the same window size or mask size, such as about a 2×3 pixelmask for detecting a distant taillight) for detecting and identifyingtaillights (or other light sources) at each distance or range, and thusmay substantially reduce the memory requirements of the processor.

Typically, if a processor is to identify a taillight that is about 200meters (or thereabouts) in front of the controlled vehicle, theprocessor may utilize a 2×3 pixel mask to determine if the detectedlight source is about the expected size of a taillight to assist incorrectly identifying the taillights. However, if the light source iscloser to the vehicle, such as at about 100 meters or thereabouts infront of the controlled vehicle, the processor would process the imagedata with a larger mask or window because the light source would belarger when it is closer to the controlled vehicle. When the lightsource is even closer, such as about 50 meters or less, to thecontrolled vehicle, an even larger window or mask is utilized toidentify the detected light source. Thus, the control system requiressufficient memory capability to store the different window sizes fordetecting the various light sources at various distances in front of thecontrolled vehicle. Such memory or data storage can be costly and thusmay add to the cost of the headlamp control system (or othervision-based system).

However, by processing the captured images at different resolutions(such as a higher resolution, a medium resolution and a lowerresolution), the system may generally equalize the sizes of the imagedobjects or light sources for the various distances from the controlledvehicle, so that only a single sized mask or window need be utilized foridentifying a particular light source, such as a taillight of a leadingvehicle. This is because a taillight at about 200 meters may take up awindow of about 2×3 pixels of a 640×480 higher resolution image, while ataillight at about 100 meters or thereabouts may take up a window ofabout 2×3 pixels of a 320×240 medium resolution image (which would beabout a 4×6 pixel window if it were a higher resolution image), and ataillight at about 50 meters or less may take up a window of about 2×3pixels of a 160×120 lower resolution image (which would be about a 8×12pixel window if it were a higher resolution image).

Thus, by processing the different resolution images, the control systemmay utilize the same mask or window or classifier for identifying adetected light source. Although the intensity of the detected lightsources would be different (such as, for example, the intensity of thelight source of the medium resolution image may be eight times theintensity of the light source of the low resolution image and theintensity of the light source of the high resolution image may be 64times the intensity of the light source of the low resolution image),this can be readily accounted for when detecting the light source andidentifying the detected light source. Thus, when a light source isdetected that may be a taillight of a leading vehicle, the processor mayprocess the image data with a single classifier (such as for classifyingthe light source as a taillight) regardless of the distance to thedetected light source from the controlled vehicle, and thus, the memoryrequirements of the system for multiple classifiers may be substantiallyreduced to reduce the cost of the control system.

Optionally, the control system may be operable to superimpose a code orflicker on the headlight beams to communicate a code or message tocontrol systems of other vehicles or of roadside monitors or stations,such as by utilizing aspects of the systems described in U.S. Pat. No.7,004,606, which is hereby incorporated herein by reference in itsentirety. For example, the headlamps (or other vehicle lights) could beused to signal other drivers with “messages” which other vehicle'smachine vision systems could decode, while typical drivers without suchsystems are unaware of the communication system. Such a code would becamouflaged to people viewing the headlamps or other lights, but visibleto the machine vision systems of the other vehicles. Different flickerrates or different color combinations or spectral signature of thelights may communicate different codes, and the codes may be presetcodes (such as, for example, a code that communicates to the driver ofthe other vehicle or vehicles that there is an accident ahead or thelike), or may be entered by the driver of the controlled vehicle (suchas via a voice input or manual input or the like).

Thus, with vehicles inside the line of sight distances, messages may besent from the controlled vehicle to other vehicles via the code embeddedin or superimposed on the output signal or illumination signal of thevehicle lights. For example, a code or message may be communicated froma controlled vehicle passing an accident to all similarly equippedoncoming traffic to alert the oncoming traffic of the accident. The codemay comprise a color change (such as a subtle color change) in the colorof the light emitted by the vehicle light source or a flicker (such as ahigh frequency flicker that is not readily noticeable or discernible toa human observer) or the like, and may be readily detected andidentified or decoded by a similar control system of another vehicle.For example, the vehicle light source may comprise yellow and blue LEDsflickering at a predetermined rate and pattern and can thus encodeinformation or data or messages which would look like a typical whiteHID or halogen headlight to human observers. Human perception offlickering in this color pair is worse than other colors which couldalso produce white, such as those nearer the red and green colors. Thisflicker rate of the yellow and blue LEDs thus may be lower than othercolor combinations (while still avoiding detection by the humanobservers), such as less than about 60 Hz.

For communicating messages or codes rearwardly, the taillights may alsoor otherwise be flickered or adjusted or coded to communicate a messageor data. Red taillight location in color space may not be optimal forflickering different colors, but using flicker rates above about 60 Hzcan provide the desired communication means while limiting orsubstantially avoiding human detection. Optionally, the light sourcesmay flicker or may have superimposed thereon an illumination output inthe infrared or near infrared range of the spectrum, where humans havepoor sensitivity, and where the imaging devices may be highly sensitive.

Optionally, the control system of the present invention may be operableto determine if a pixel or pixels of the imaging array is eitherinoperable or “bad” or blocked, so that the control system may ignorethe bad/blocked pixel output to avoid adversely affecting averages ofpixel output intensities during the image processing. The bad pixeldetection process or algorithm may be performed periodically when thesystem is operating. For example, a captured frame or image may bededicated to bad pixel detection. If a bad pixel or pixels is/aredetected, averaging of the output intensities of the pixels surroundingthe bad pixel may be performed to accommodate or replace the bad orinoperable pixel.

It is further envisioned that the control system may be operable todetermine if some or all of the pixels of the imaging array are blocked(such as via an object or dirt or debris at the vehicle windshield orthe like) and to adapt the image processing accordingly or notify oralert the driver of the vehicle that such blockage has occurred. Forexample, a partial or total day blockage algorithm may be run duringdaytime lighting conditions, such as in response to a user input or ondemand, while a partial or total night blockage algorithm may be runwhen the ambient condition is indicative of nighttime lightingconditions. When the total blockage algorithm is run, the number ofpixels above an intensity threshold may be counted for a captured imageor frame, and if, over a number of frames, the count of the brightpixels is continuously below a threshold, the control system mayconclude that the imaging device substantially or totally blocked. Whenthe partial blockage algorithm is run, the control system may performregion-based processing to take into account intensity variations indifferent regions of the pixelated imaging array. Based on intensityvariations with neighboring or adjacent regions and the continuity ofthe variations over time, the control may determine that the imagingarray is partially blocked. The control system may process the blockedpixel region in a night mode to reduce or substantially preclude thepossibility of a false blockage detection.

If either partial or total blockage is detected or determined, thesystem may adapt the image processing to accommodate the blocked pixels,or the system may alert the driver of the vehicle that the pixels areblocked so that the driver or user may unblock the imaging device (suchas via cleaning the windshield of the vehicle), or the system mayactuate the vehicle windshield wipers to clear the windshield at theimaging device or the like, or the system may actuate a blower system(such as a defogger system or the like) of the vehicle to direct orforce or blow air toward the detected blockage to clear the windshieldor window or area in the forward field of view of the imaging device.Optionally, the control thus may detect that at least a portion of theimaging device or photosensor array is blocked and may switch to a lowbeam mode in response to the detection (so as to allow the system toconfirm the existence of the blockage without the high beams on duringthis period of time), and the system may at least one of (a) alert thedriver of the subject vehicle of the detected blockage so that he or shecan clean the windshield or sensor or otherwise remove the blockage oractuate the wipers and/or related system of the vehicle to remove theblockage; (b) automatically actuate a wiper (such as the windshieldwipers) of the vehicle to remove the blockage from the forward field ofview of the imaging device; and (c) automatically actuate a blowersystem of the vehicle to remove or dissipate the blockage from theforward field of view. The system or control may also detect that theblockage has been removed from the forward field of view and may resumethe normal functionality of the headlamp control system and/or the wipersystem of the vehicle and/or the blower system of the vehicle.

Optionally, the imaging sensor (and/or aspects of the control systemdescribed above) may be suitable for use in connection with othervehicle imaging systems, such as, for example, a blind spot detectionsystem, where a blind spot indicator may be operable to provide anindication to the driver of the host vehicle that an object or othervehicle has been detected in the lane or area adjacent to the side ofthe host vehicle. In such a blind spot detector/indicator system, theblind spot detection system may include an imaging sensor or sensors, orultrasonic sensor or sensors, or sonar sensor or sensors or the like.For example, the blind spot detection system may utilize aspects of theblind spot detection and/or imaging systems described in U.S. Pat. Nos.7,038,577; 6,882,287; 6,198,409; 5,929,786 and/or 5,786,772, and/or U.S.patent application Ser. No. 11/315,675, filed Dec. 22, 2005, andpublished Aug. 17, 2006 as U.S. Publication No. US-2006-0184297; and/orSer. No. 11/239,980, filed Sep. 30, 2005, and/or U.S. provisionalapplications, Ser. No. 60/696,953, filed Jul. 6, 2005; Ser. No.60/628,709, filed Nov. 17, 2004; Ser. No. 60/614,644, filed Sep. 30,2004; and/or Ser. No. 60/618,686, filed Oct. 14, 2004, and/or PCTApplication No. PCT/US2006/026148, filed Jul. 5, 2006, and publishedJan. 11, 2007 as PCT Publication No. WO 2007/005942, and/or of thereverse or backup aid systems, such as the rearwardly directed vehiclevision systems described in U.S. Pat. Nos. 5,550,677; 5,760,962;5,670,935; 6,201,642; 6,396,397; 6,498,620; 6,717,610; 6,757,109 and/or7,005,974, and/or of the rain sensors described in U.S. Pat. Nos.6,250,148 and 6,341,523, and/or of other imaging systems, such as thetypes described in U.S. Pat. Nos. 7,123,168; 6,353,392 and/or 6,313,454,with all of the above referenced U.S. patents, patent applications andprovisional applications and PCT applications being commonly assignedand being hereby incorporated herein by reference in their entireties.

Optionally, the optical system may be held by features of a housingassembly of an interior rearview mirror assembly of an accessory moduleor the like. The housing assembly may utilize aspects of the modules orassemblies described in U.S. Pat. Nos. 7,004,593; 6,968,736; 6,877,888;6,824,281; 6,690,268; 6,672,744; 6,593,565; 6,516,664; 6,501,387;6,428,172; 6,386,742; 6,341,523; 6,329,925; and 6,326,613; 6,250,148 and6,124,886, and/or U.S. patent application Ser. No. 10/538,724, filedJun. 13, 2005 and published Mar. 9, 2006 as U.S. patent publication No.US2006-0050018, and/or Ser. No. 11/201,661, filed Aug. 11, 2005, nowU.S. Pat. No. 7,480,149, and/or PCT Application No. PCT/US03/40611,filed Dec. 19, 2003; PCT Application No. PCT/US03/03012, filed Jan. 31,2003, and/or PCT Application No. PCT/US04/15424, filed May 18, 2004,and/or Ireland pat. applications, Ser. No. S2004/0614, filed Sep. 15,2004; Ser. No. S2004/0838, filed Dec. 14, 2004; and Ser. No. S2004/0840,filed Dec. 15, 2004, which are all hereby incorporated herein byreference in their entireties.

Optionally, the mirror assembly and/or accessory module or windshieldelectronics module may include one or more displays, such as fordisplaying the captured images or video images captured by the imagingsensor or sensors of the vehicle, such as the displays of the typesdisclosed in U.S. Pat. Nos. 7,004,593; 5,530,240 and/or 6,329,925, whichare hereby incorporated herein by reference, and/or display-on-demand ortransflective type displays, such as the types disclosed in U.S. Pat.Nos. 7,195,381; 6,690,268; 5,668,663 and/or 5,724,187, and/or in U.S.patent application Ser. No. 11/021,065, filed Dec. 23, 2004, now U.S.Pat. No. 7,255,451; Ser. No. 10/528,269, filed Mar. 17, 2005, now U.S.Pat. No. 7,274,501; Ser. No. 10/533,762, filed May 4, 2005, now U.S.Pat. No. 7,184,190; Ser. No. 10/538,724, filed Jun. 13, 2005 andpublished Mar. 9, 2006 as U.S. patent publication No. US2006-0050018;Ser. No. 11/226,628, filed Sep. 14, 2005 and published Mar. 23, 2006 asU.S. patent publication No. US 2006-0061008; Ser. No. 10/993,302, filedNov. 19, 2004, now U.S. Pat. No. 7,338,177; and/or Ser. No. 11/284,543,filed Nov. 22, 2005, now U.S. Pat. No. 7,370,983, and/or PCT ApplicationNo. PCT/US03/29776, filed Sep. 9, 2003; and/or PCT Application No.PCT/US03/35381, filed Nov. 5, 2003, and/or PCT Application No.PCT/US03/40611, filed Dec. 19, 2003, and/or PCT Application No.PCT/US2006/018567, filed May 15, 2006, which are all hereby incorporatedherein by reference, or may include or incorporate video displays or thelike, such as the types described in PCT Application No. PCT/US03/40611,filed Dec. 19, 2003, and/or U.S. patent application Ser. No. 10/538,724,filed Jun. 13, 2005 and published Mar. 9, 2006 as U.S. patentpublication No. US2006-0050018; and/or Ser. No. 11/284,543, filed Nov.22, 2005, now U.S. Pat. No. 7,370,983, and/or U.S. provisionalapplications, Ser. No. 60/732,245, filed Nov. 1, 2005; Ser. No.60/759,992, filed Jan. 18, 2006; and/or Ser. No. 60/836,219, filed Aug.8, 2006, which are hereby incorporated herein by reference.

The imaging sensor may be incorporated at or in an accessory module orwindshield electronics module (such as described above), or may beincorporated at or in an interior rearview mirror assembly of thevehicle, while remaining within the spirit and scope of the presentinvention. Optionally, the mirror assembly and/or module may support oneor more other accessories or features, such as one or more electrical orelectronic devices or accessories. For example, illumination sources orlights, such as map reading lights or one or more other lights orillumination sources, such as illumination sources of the typesdisclosed in U.S. Pat. Nos. 7,195,381; 6,690,268; 5,938,321; 5,813,745;5,820,245; 5,673,994; 5,649,756; 5,178,448; 5,671,996; 4,646,210;4,733,336; 4,807,096; 6,042,253; 6,971,775 and/or 5,669,698, and/or U.S.patent application Ser. No. 10/933,842, filed Sep. 3, 2004, now U.S.Pat. No. 7,249,860, which are hereby incorporated herein by reference,may be included in the mirror assembly or module. The illuminationsources and/or the circuit board may be connected to one or more buttonsor inputs for activating and deactivating the illumination sources.Optionally, the mirror assembly or module may also or otherwise includeother accessories, such as microphones, such as analog microphones ordigital microphones or the like, such as microphones of the typesdisclosed in U.S. Pat. Nos. 6,243,003; 6,278,377 and/or 6,420,975,and/or in PCT Application No. PCT/US03/308877, filed Oct. 1, 2003.Optionally, the mirror assembly may also or otherwise include otheraccessories, such as a telematics system, speakers, antennas, includingglobal positioning system (GPS) or cellular phone antennas, such asdisclosed in U.S. Pat. No. 5,971,552, a communication module, such asdisclosed in U.S. Pat. No. 5,798,688, a voice recorder, transmittersand/or receivers, such as for a garage door opener or a vehicle doorunlocking system or the like (such as a remote keyless entry system), adigital network, such as described in U.S. Pat. No. 5,798,575, a memorymirror system, such as disclosed in U.S. Pat. No. 5,796,176, ahands-free phone attachment, a video device for internal cabinsurveillance (such as for sleep detection or driver drowsiness detectionor the like) and/or video telephone function, such as disclosed in U.S.Pat. Nos. 5,760,962 and/or 5,877,897, a remote keyless entry receiver, aseat occupancy detector, a remote starter control, a yaw sensor, aclock, a carbon monoxide detector, status displays, such as displaysthat display a status of a door of the vehicle, a transmission selection(4 wd/2 wd or traction control (TCS) or the like), an antilock brakingsystem, a road condition (that may warn the driver of icy roadconditions) and/or the like, a trip computer, a tire pressure monitoringsystem (TPMS) receiver (such as described in U.S. Pat. Nos. 6,124,647;6,294,989; 6,445,287; 6,472,979 and/or 6,731,205; and/or U.S. patentapplication Ser. No. 11/232,324, filed Sep. 21, 2005, now U.S. Pat. No.7,423,522, and/or an ONSTAR® system and/or any other accessory orcircuitry or the like (with all of the above-referenced patents and PCTand U.S. patent applications being commonly assigned, and with thedisclosures of the referenced patents and patent applications beinghereby incorporated herein by reference in their entireties).

Optionally, the mirror assembly or module may include one or more userinputs for controlling or activating/deactivating one or more electricalaccessories or devices of or associated with the mirror assembly ormodule or vehicle. The mirror assembly or module may comprise any typeof switches or buttons, such as touch or proximity sensing switches,such as touch or proximity switches of the types described in PCTApplication No. PCT/US03/40611, filed Dec. 19, 2003; and/or U.S. Pat.Nos. 6,001,486; 6,310,611; 6,320,282 and 6,627,918; and/or U.S. patentapplication Ser. No. 09/817,874, filed Mar. 26, 2001, now U.S. Pat. No.7,224,324; Ser. No. 10/956,749, filed Oct. 1, 2004, now U.S. Pat. No.7,446,924; Ser. No. 10/933,842, filed Sep. 3, 2004, now U.S. Pat. No.7,249,860; Ser. No. 11/021,065, filed Dec. 23, 2004, now U.S. Pat. No.7,255,451; and/or Ser. No. 11/140,396, filed May 27, 2005, now U.S. Pat.No. 7,360,932, which are hereby incorporated herein by reference, or theinputs may comprise other types of buttons or switches, such as thosedescribed in U.S. patent application Ser. No. 11/029,695, filed Jan. 5,2005, now U.S. Pat. No. 7,253,723; and/or Ser. No. 11/451,639, filedJun. 13, 2006, now U.S. Pat. No. 7,527,403, which are herebyincorporated herein by reference, or such as fabric-made positiondetectors, such as those described in U.S. Pat. Nos. 6,504,531;6,501,465; 6,492,980; 6,452,479; 6,437,258 and 6,369,804, which arehereby incorporated herein by reference. Other types of switches orbuttons or inputs or sensors may be incorporated to provide the desiredfunction, without affecting the scope of the present invention.

Optionally, any such user inputs or buttons may comprise user inputs fora garage door opening system, such as a vehicle based garage dooropening system of the types described in U.S. Pat. Nos. 6,396,408;6,362,771 and 5,798,688, and/or U.S. patent application Ser. No.10/770,736, filed Feb. 3, 2004, now U.S. Pat. No. 7,023,322; and/or U.S.provisional applications, Ser. No. 60/502,806, filed Sep. 12, 2003; andSer. No. 60/444,726, filed Feb. 4, 2003, which are hereby incorporatedherein by reference. The user inputs may also or otherwise function toactivate and deactivate a display or function or accessory, and/or mayactivate/deactivate and/or commence a calibration of a compass system ofthe mirror assembly and/or vehicle. The compass system may includecompass sensors and circuitry within the mirror assembly or within acompass pod or module at or near or associated with the mirror assembly.Optionally, the user inputs may also or otherwise comprise user inputsfor a telematics system of the vehicle, such as, for example, an ONSTAR®system as found in General Motors vehicles and/or such as described inU.S. Pat. Nos. 4,862,594; 4,937,945; 5,131,154; 5,255,442; 5,632,092;5,798,688; 5,971,552; 5,924,212; 6,243,003; 6,278,377; 6,420,975;6,946,978; 6,477,464; 6,678,614 and/or 7,004,593, and/or U.S. patentapplication Ser. No. 10/645,762, filed Aug. 20, 2003, now U.S. Pat. No.7,167,796; and Ser. No. 10/964,512, filed Oct. 13, 2004, now U.S. Pat.No. 7,308,341; and/or PCT Application No. PCT/US03/40611, filed Dec. 19,2003, and/or PCT Application No. PCT/US03/308877, filed Oct. 1, 2003,which are all hereby incorporated herein by reference.

Optionally, the accessory module may utilize aspects of other accessorymodules or windshield electronics modules or the like, such as the typesdescribed in U.S. patent application Ser. No. 10/958,087, filed Oct. 4,2004, now U.S. Pat. No. 7,188,963; and/or Ser. No. 11/201,661, filedAug. 11, 2005, now U.S. Pat. No. 7,480,149, and/or U.S. Pat. Nos.7,004,593; 6,824,281; 6,690,268; 6,250,148; 6,341,523; 6,593,565;6,428,172; 6,501,387; 6,329,925 and 6,326,613, and/or in PCT ApplicationNo. PCT/US03/40611, filed Dec. 19, 2003, and/or Ireland pat.applications, Ser. No. S2004/0614, filed Sep. 15, 2004; Ser. No.S2004/0838, filed Dec. 14, 2004; and Ser. No. S2004/0840, filed Dec. 15,2004, which are all hereby incorporated herein by reference.

The reflective element of the rearview mirror assembly of the vehiclemay comprise an electro-optic or electrochromic reflective element orcell, such as an electrochromic mirror assembly and electrochromicreflective element utilizing principles disclosed in commonly assignedU.S. Pat. Nos. 7,195,381; 6,690,268; 5,140,455; 5,151,816; 6,178,034;6,154,306; 6,002,544; 5,567,360; 5,525,264; 5,610,756; 5,406,414;5,253,109; 5,076,673; 5,073,012; 5,117,346; 5,724,187; 5,668,663;5,910,854; 5,142,407 and/or 4,712,879, and/or U.S. patent applicationSer. No. 11/021,065, filed Dec. 23, 2004, now U.S. Pat. No. 7,255,451;Ser. No. 11/226,628, filed Sep. 14, 2005 and published Mar. 23, 2006 asU.S. patent publication No. US 2006-0061008, and/or PCT PatentApplication No. PCT/US2006/018567, filed May 15, 2006, which are allhereby incorporated herein by reference, and/or as disclosed in thefollowing publications: N. R. Lynam, “Electrochromic AutomotiveDay/Night Mirrors”, SAE Technical Paper Series 870636 (1987); N. R.Lynam, “Smart Windows for Automobiles”, SAE Technical Paper Series900419 (1990); N. R. Lynam and A. Agrawal, “Automotive Applications ofChromogenic Materials”, Large Area Chromogenics: Materials and Devicesfor Transmittance Control, C. M. Lampert and C. G. Granquist, EDS.,Optical Engineering Press, Wash. (1990), which are hereby incorporatedby reference herein. The thicknesses and materials of the coatings onthe substrates of the electrochromic reflective element, such as on thethird surface of the reflective element assembly, may be selected toprovide a desired color or tint to the mirror reflective element, suchas a blue colored reflector, such as is known in the art and/or such asdescribed in U.S. Pat. Nos. 5,910,854 and 6,420,036, and in PCTApplication No. PCT/US03/29776, filed Sep. 9, 2003, which are all herebyincorporated herein by reference.

Optionally, use of an elemental semiconductor mirror, such as a siliconmetal mirror, such as disclosed in U.S. Pat. Nos. 6,286,965; 6,196,688;5,535,056; 5,751,489 and 6,065,840, and/or in U.S. patent applicationSer. No. 10/993,302, filed Nov. 19, 2004, now U.S. Pat. No. 7,338,177,which are all hereby incorporated herein by reference, can beadvantageous because such elemental semiconductor mirrors (such as canbe formed by depositing a thin film of silicon) can be greater than 50percent reflecting in the photopic (SAE J964a measured), while beingalso substantially transmitting of light (up to 20 percent or evenmore). Such silicon mirrors also have the advantage of being able to bedeposited onto a flat glass substrate and to be bent into a curved (suchas a convex or aspheric) curvature, which is also advantageous sincemany passenger-side exterior rearview mirrors are bent or curved.

Optionally, the reflective element may include a perimeter metallicband, such as the types described in PCT Application No. PCT/US03/29776,filed Sep. 19, 2003; and/or PCT Application No. PCT/US03/35381, filedNov. 5, 2003; and/or U.S. patent application Ser. No. 11/021,065, filedDec. 23, 2004, now U.S. Pat. No. 7,255,451; and/or Ser. No. 11/226,628,filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S. patentpublication No. US 2006-0061008, which are hereby incorporated herein byreference. Optionally, the reflective element may include indicia formedat and viewable at the reflective element, such as by utilizing aspectsof the reflective elements described in PCT Patent Application No.PCT/US2006/018567, filed May 15, 2006, which are hereby incorporatedherein by reference.

Optionally, the reflective element of the mirror assembly may comprise asingle substrate with a reflective coating at its rear surface, withoutaffecting the scope of the present invention. The mirror assembly thusmay comprise a prismatic mirror assembly or other mirror having a singlesubstrate reflective element, such as a mirror assembly utilizingaspects described in U.S. Pat. Nos. 6,318,870; 6,598,980; 5,327,288;4,948,242; 4,826,289; 4,436,371 and 4,435,042; and PCT Application No.PCT/US04/015424, filed May 18, 2004; and U.S. patent application Ser.No. 10/933,842, filed Sep. 3, 2004, now U.S. Pat. No. 7,249,860, whichare hereby incorporated herein by reference. Optionally, the reflectiveelement may comprise a conventional prismatic or flat reflective elementor prism, or may comprise a prismatic or flat reflective element of thetypes described in PCT Application No. PCT/US03/29776, filed Sep. 19,2003; U.S. patent application Ser. No. 10/709,434, filed May 5, 2004,now U.S. Pat. No. 7,420,756; Ser. No. 10/933,842, filed Sep. 3, 2004,now U.S. Pat. No. 7,249,860; Ser. No. 11/021,065, filed Dec. 23, 2004,now U.S. Pat. No. 7,255,451; and/or Ser. No. 10/993,302, filed Nov. 19,2004, now U.S. Pat. No. 7,338,177, and/or PCT Application No.PCT/US2004/015424, filed May 18, 2004, which are all hereby incorporatedherein by reference, without affecting the scope of the presentinvention.

Changes and modifications to the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited by the scope of the appendedclaims as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

1. A vehicular imaging system, said vehicular imaging system comprising: a camera comprising a CMOS imaging sensor; said imaging sensor comprising a photosensor array having a plurality of photosensor elements; wherein said camera is disposed in a vehicle equipped with said vehicular imaging system, and wherein said camera is disposed behind a windshield of the equipped vehicle and views through a portion of the windshield; wherein said camera views a scene forward of the equipped vehicle; wherein the portion of the windshield comprises a portion cleaned by a windshield wiper of the equipped vehicle; wherein said camera is operable to capture frames of image data at an image frame capture rate that is between 5 times per second and 120 times per second; wherein image data captured by said camera is provided to a control, said control comprising an image processor; wherein said control receives, via a communication bus of the equipped vehicle, at least one selected from the group consisting of (i) vehicle pitch information relating to pitch of the equipped vehicle, (ii) vehicle yaw information relating to yaw of the equipped vehicle and (iii) vehicle steering information relating to steering of the equipped vehicle; wherein said vehicular imaging system automatically corrects for misalignment of said camera disposed behind the windshield of the equipped vehicle and viewing through the portion of the windshield; and wherein image data captured by said camera is processed at said control for a lane departure warning system of the equipped vehicle and for at least one selected from the group consisting of (i) an automatic headlamp control system of the equipped vehicle, (ii) a collision avoidance system of the equipped vehicle and (iii) an adaptive front lighting system of the equipped vehicle.
 2. The vehicular imaging system of claim 1, wherein said vehicular imaging system automatically corrects for misalignment of said camera of up to +/−4 degrees from an aligned condition of said camera.
 3. The vehicular imaging system of claim 2, wherein said control corrects a misalignment of said camera that occurs due to use of the equipped vehicle.
 4. The vehicular imaging system of claim 3, wherein the misalignment that occurs due to use of the equipped vehicle arises from at least one selected from the group consisting of (i) a change in pitch of the equipped vehicle, (ii) a change in tilt of the equipped vehicle and (iii) a change in yaw of the equipped vehicle.
 5. The vehicular imaging system of claim 3, wherein the misalignment arises from a load condition of the equipped vehicle.
 6. The vehicular imaging system of claim 1, wherein said control is operable to process a reduced set of image data more than other image data.
 7. The vehicular imaging system of claim 1, wherein said control processes captured image data at a higher resolution level of processing of captured image data and at a lower resolution level of processing of captured image data.
 8. The vehicular imaging system of claim 7, wherein said control processes captured image data at the lower resolution level responsive to distance between the equipped vehicle and an object viewed by said camera being below a threshold level for the object, and wherein said control processes captured image data at the higher resolution level responsive to distance between the equipped vehicle and the object viewed by said camera being above the threshold level for the object.
 9. The vehicular imaging system of claim 1, wherein said vehicular imaging system detects when at least a portion of said photosensor array of said imaging sensor of said camera is blocked.
 10. The vehicular imaging system of claim 9, wherein said vehicular imaging system determines that said photosensor array of said camera is totally blocked when, over a plurality of frames of image data captured by said camera, processing of image data at said control determines that a count of bright pixels remains below a threshold.
 11. The vehicular imaging system of claim 9, wherein said vehicular imaging system determines that said photosensor array of said camera is partially blocked via said control performing region-based image processing to take into account intensity variations in different regions of said photosensor array of said camera.
 12. The vehicular imaging system of claim 1, wherein said vehicular imaging system automatically detects a driving side of a road being traveled along by the equipped vehicle.
 13. The vehicular imaging system of claim 12, wherein image processing at said control of image data captured by said camera identifies a taillight of another vehicle that is 200 meters in front of the equipped vehicle.
 14. The vehicular imaging system of claim 1, wherein said vehicular imaging system is operable to encode signals into a light output of a light of the equipped vehicle to allow an imaging system of another vehicle to detect an encoded signal.
 15. The vehicular imaging system of claim 1, wherein said vehicular imaging system is operable to detect and ameliorate a bad photosensor element of said plurality of photosensor elements of said imaging sensor of said camera.
 16. The vehicular imaging system of claim 1, wherein a reduced image data set of image data captured by said camera is processed at said control, the reduced image data set being representative of a portion of the captured image data as captured by a particular grouping of said photosensor elements of said imaging sensor of said camera.
 17. The vehicular imaging system of claim 1, wherein the automatic correction for misalignment of said camera mounted at the equipped vehicle comprises image processing at said control of image data captured by said camera determining where converging road features of a road along which the equipped vehicle is traveling would converge.
 18. The vehicular imaging system of claim 1, wherein the automatic correction for misalignment of said camera comprises comparing an imaged location of an object present in a field of view of said imaging sensor to an expected location of the object.
 19. The vehicular imaging system of claim 1, wherein the automatic correction for misalignment of said camera mounted at the equipped vehicle comprises image processing at said control of image data captured by said camera comparing an imaged location of an object viewed by said camera to an expected location of the object, wherein the imaged location is determined by image processing at said control of image data captured said camera and wherein the expected location is determined responsive to vehicle data carried to said control via said communication bus.
 20. A vehicular imaging system, said vehicular imaging system comprising: a camera comprising a CMOS imaging sensor; said imaging sensor comprising a photosensor array having a plurality of photosensor elements; wherein said camera is disposed in a vehicle equipped with said vehicular imaging system, and wherein said camera is disposed behind a windshield of the equipped vehicle and views through a portion of the windshield; wherein said camera views a scene forward of the equipped vehicle; wherein the portion of the windshield comprises a portion cleaned by a windshield wiper of the equipped vehicle; wherein said camera is operable to capture frames of image data at an image frame capture rate that is between 5 times per second and 120 times per second; wherein image data captured by said camera is provided to a control, said control comprising an image processor; wherein said control receives, via a communication bus of the equipped vehicle, at least one selected from the group consisting of (i) vehicle pitch information relating to pitch of the equipped vehicle, (ii) vehicle yaw information relating to yaw of the equipped vehicle and (iii) vehicle steering information relating to steering of the equipped vehicle; wherein said vehicular imaging system automatically corrects for misalignment of said camera disposed behind the windshield of the equipped vehicle and viewing through the portion of the windshield; wherein said vehicular imaging system automatically corrects for misalignment of said camera of up to +/−4 degrees from an aligned condition of said camera; wherein the automatic correction for misalignment of said camera mounted at the equipped vehicle comprises image processing at said control of image data captured by said camera determining where converging road features of a road along which the equipped vehicle is traveling would converge; and wherein image data captured by said camera is processed at said control for a lane departure warning system of the equipped vehicle and for at least one selected from the group consisting of (i) an automatic headlamp control system of the equipped vehicle, (ii) a collision avoidance system of the equipped vehicle and (iii) an adaptive front lighting system of the equipped vehicle.
 21. The vehicular imaging system of claim 20, wherein image processing at said control of image data captured by said camera identifies a taillight of another vehicle that is 200 meters in front of the equipped vehicle.
 22. The vehicular imaging system of claim 21, wherein said control corrects a misalignment of said camera that occurs due to use of the equipped vehicle.
 23. The vehicular imaging system of claim 22, wherein the misalignment that occurs due to use of the equipped vehicle arises from at least one selected from the group consisting of (i) a change in pitch of the equipped vehicle, (ii) a change in tilt of the equipped vehicle and (iii) a change in yaw of the equipped vehicle.
 24. The vehicular imaging system of claim 20, wherein said control is operable to process a reduced set of image data more than other image data.
 25. The vehicular imaging system of claim 24, wherein said control processes captured image data at a higher resolution level of processing of captured image data and at a lower resolution level of processing of captured image data.
 26. The vehicular imaging system of claim 25, wherein said control processes captured image data at the lower resolution level responsive to distance between the equipped vehicle and an object viewed by said camera being below a threshold level for the object, and wherein said control processes captured image data at the higher resolution level responsive to distance between the equipped vehicle and the object viewed by said camera being above the threshold level for the object.
 27. The vehicular imaging system of claim 20, wherein said vehicular imaging system detects when at least a portion of said photosensor array of said imaging sensor of said camera is blocked.
 28. The vehicular imaging system of claim 27, wherein said vehicular imaging system determines that said photosensor array of said camera is totally blocked when, over a plurality of frames of image data captured by said camera, processing of image data at said control determines that a count of bright pixels remains below a threshold.
 29. The vehicular imaging system of claim 27, wherein said vehicular imaging system determines that said photosensor array of said camera is partially blocked via said control performing region-based image processing to take into account intensity variations in different regions of said photosensor array of said camera.
 30. The vehicular imaging system of claim 20, wherein said vehicular imaging system automatically detects a driving side of the road being traveled along by the equipped vehicle.
 31. The vehicular imaging system of claim 30, wherein a reduced image data set of image data captured by said camera is processed at said control, the reduced image data set being representative of a portion of the captured image data as captured by a particular grouping of said photosensor elements of said imaging sensor of said camera.
 32. A vehicular imaging system, said vehicular imaging system comprising: a camera comprising a CMOS imaging sensor; said imaging sensor comprising a photosensor array having a plurality of photosensor elements; wherein said camera is disposed in a vehicle equipped with said vehicular imaging system, and wherein said camera is disposed behind a windshield of the equipped vehicle and views through a portion of the windshield; wherein said camera views a scene forward of the equipped vehicle; wherein the portion of the windshield comprises a portion cleaned by a windshield wiper of the equipped vehicle; wherein said camera is operable to capture frames of image data at an image frame capture rate that is between 5 times per second and 120 times per second; wherein image data captured by said camera is provided to a control, said control comprising an image processor; wherein said control receives, via a communication bus of the equipped vehicle, at least one selected from the group consisting of (i) vehicle pitch information relating to pitch of the equipped vehicle, (ii) vehicle yaw information relating to yaw of the equipped vehicle and (iii) vehicle steering information relating to steering of the equipped vehicle; wherein said vehicular imaging system automatically corrects for misalignment of said camera disposed behind the windshield of the equipped vehicle and viewing through the portion of the windshield; wherein said vehicular imaging system automatically corrects for misalignment of said camera of up to +/−4 degrees from an aligned condition of said camera; wherein the automatic correction for misalignment of said camera mounted at the equipped vehicle comprises image processing at said control of image data captured by said camera comparing an imaged location of an object viewed by said camera to an expected location of the object; wherein the imaged location is determined by image processing at said control of image data captured said camera; wherein the expected location is determined responsive to vehicle data carried to said control via said communication bus; and wherein image data captured by said camera is processed at said control for a lane departure warning system of the equipped vehicle and for at least one selected from the group consisting of (i) an automatic headlamp control system of the equipped vehicle, (ii) a collision avoidance system of the equipped vehicle and (iii) an adaptive front lighting system of the equipped vehicle.
 33. The vehicular imaging system of claim 32, wherein image processing at said control of image data captured by said camera identifies a taillight of another vehicle that is 200 meters in front of the equipped vehicle.
 34. The vehicular imaging system of claim 33, wherein said vehicular imaging system automatically detects a driving side of a road being traveled along by the equipped vehicle.
 35. The vehicular imaging system of claim 34, wherein a reduced image data set of image data captured by said camera is processed at said control, the reduced image data set being representative of a portion of the captured image data as captured by a particular grouping of said photosensor elements of said imaging sensor of said camera.
 36. The vehicular imaging system of claim 33, wherein said control is operable to process a reduced set of image data more than other image data.
 37. The vehicular imaging system of claim 32, wherein said control corrects a misalignment of said camera that occurs due to use of the equipped vehicle.
 38. The vehicular imaging system of claim 37, wherein the misalignment that occurs due to use of the equipped vehicle arises from at least one selected from the group consisting of (i) a change in pitch of the equipped vehicle, (ii) a change in tilt of the equipped vehicle and (iii) a change in yaw of the equipped vehicle.
 39. The vehicular imaging system of claim 32, wherein said control processes captured image data at a higher resolution level of processing of captured image data and at a lower resolution level of processing of captured image data.
 40. The vehicular imaging system of claim 39, wherein said control processes captured image data at the lower resolution level responsive to distance between the equipped vehicle and an object viewed by said camera being below a threshold level for the object, and wherein said control processes captured image data at the higher resolution level responsive to distance between the equipped vehicle and the object viewed by said camera being above the threshold level for the object.
 41. The vehicular imaging system of claim 32, wherein said vehicular imaging system detects when at least a portion of said photosensor array of said imaging sensor of said camera is blocked.
 42. The vehicular imaging system of claim 41, wherein said vehicular imaging system determines that said photosensor array of said camera is totally blocked when, over a plurality of frames of image data captured by said camera, processing of image data at said control determines that a count of bright pixels remains below a threshold.
 43. The vehicular imaging system of claim 41, wherein said vehicular imaging system determines that said photosensor array of said camera is partially blocked via said control performing region-based image processing to take into account intensity variations in different regions of said photosensor array of said camera. 